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PREVENTABLE FAILURE 

Research Summary 

In 2007, spurred by University of Chicago Consortium on Chicago 
School Research (UChicago CCSR) research reports, leadership at 
the Chicago Public Schools (CPS) began a new targeted approach to 
reducing course failure in the ninth grade. The research suggested that 
the transition between eighth and ninth grade played a critical role in 
shaping students’ long-term outcomes, and demonstrated that students’ 
attendance and course performance in the ninth grade year were highly 
predictive of high school dropout. 


Schools across Chicago initiated new strategies to 
improve that transition, making use of new data reports 
for monitoring and supporting students in real time, as 
well as ninth grade coordinators, and summer transi- 
tion programs. On its face this does not seem like an 
approach that would produce a system shift in perfor- 
mance, redefine approaches to school dropout, and 
call into question the conventional wisdom that urban 
neighborhood high schools could not make radical im- 
provements. And yet CPS’s focus on on-track achieved 
all of this. It moved an entire system, including many 
traditional neighborhood high schools that had been 
condemned as dropout factories. This is a summary 
of an upcoming UChicago CCSR research report that 
examines the dramatic improvements in the number 
of CPS ninth-graders who were on- track to graduation 
and the impact these improvements had on students’ 
long-term outcomes. 

Developing effective approaches to reducing drop- 
out rates is one of the highest priorities in education 
today. Despite weak economic prospects for young 
adults without high school diplomas,’ one-fourth of 
adolescents in the United States still do not make it to 
graduation.^ Graduation rates in urban districts are 
particularly low.^ Furthermore, gaps in educational 
attainment by income have actually grown in recent 
decades.'* A focus on high school graduation could help 
reverse this trend. In fact, a recent report by RAND 


on the minority attainment gap concluded that among 
the available policy options for increasing educational 
attainment, equalizing high school graduation rates 
between white and Hispanic and white and African 
American students would have the highest cost-benefit 
ratio of any strategy.® 

Urban educators are under increasing pressure to 
increase high school graduation rates, while simulta- 
neously raising student achievement for all students. 
However, educators are seldom provided with concrete 
guidance on what they could do that would make an 
immediate difference in the success of their students. 
For example, principals reading the Institute for 
Education Science’s (lES) What Works Clearinghouse 
(WWC) report on dropout prevention— intended to 
summarize best practices and identify programs with 
strong evidence of effects— would find that most of the 
dropout prevention programs provide little guidance of 
how to systematically improve their school practices. 
The majority of the seven interventions with evidence 
of positive effects are alternative schools, whole school 
change models, or after school programs. None of these 
approaches provides a principal with any strategies 
that they could use in the management of schools or 
with their teachers, short of buying a special program. 
One practice that the WWC report recommends, but for 
which there was yet little evidence when it was written, 
is that schools use data systems to accurately diagnose 
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their dropout rates and identify students at risk of drop- 
ping out. This practice was adopted by Chicago schools; 
in 2008 schools started using data reports to identify 
students at risk of dropping out in ninth grade. 

Dropout Prevention and 
the On-Track Indicator 

Over the past several years, CPS has developed a targeted 
approach to dropout prevention focused on improving 
students’ ninth grade course performance. The ap- 
proach stems largely from a series of studies published 
by UChicago CCSR researchers that narrowed down the 
myriad factors contributing to dropout to a single, man- 
ageable intervention point; the ninth grade transition. 
These studies found that as students transition to high 
school, their grades, attendance, and engagement in school 
decline markedly. Course failure becomes common, even 
among students with strong grades and test scores in 
eighth grade. These studies have strongly suggested that if 
adolescents can make the transition to high school effec- 
tively, they will be more likely to progress in subsequent 
grades and ultimately graduate from high school on time. 

In Chicago, much of the work on ninth grade co- 
alesced around the “on-track” indicator, developed by 
UChicago CCSR in the late 1990s. The on-track indica- 
tor provides a simple quantitative measure of whether 
ninth-graders are making adequate progress to gradu- 
ation based on their credit completion and course 
failures. Specifically, a student is considered “on-track” 
if she has enough credits to be promoted to tenth grade 
and had earned no more than one semester F in a core 
course. Students who end their ninth grade year on- 
track are almost four times more likely to graduate 


from high school than those who are off-track. In fact, 
a student’s on-track status is more predictive of high 
school graduation than their race/ethnicity, level of 
poverty, or test scores. The on-track indicator focused 
attention on a key developmental transition with a 
quantitative measure that could be easily calculated, 
monitored, and ultimately acted and improved upon. 

In 2007, following the release of a key report by 
UChicago CCSR on strategies affecting on- track rates, the 
district began a major initiative that promoted the use 
of data to identify students’ level of dropout risk in high 
school and intervene before students fell too far behind. 
One of the key features of this initiative was the develop- 
ment of data reports that allowed high school adminis- 
trators and teachers to monitor student performance in 
real time and identify students at risk. High schools were 
given substantial flexibility in how they used these data. 
The range of strategies and approaches developed around 
these data at both the district and high school levels is one 
of the most distinctive features of the effort. 

The effort appears to have paid substantial divi- 
dends: between 2007 and 2013, the CPS on-track rate 
rose 25 percentage points, from 57 to 82 percent (see 
Figure 1). This represents an estimated 6,900 additional 
students who finish ninth grade each year without 
significant course failures and with sufficient credits to 
earn sophomore standing. Importantly, these improve- 
ments occurred across all racial/ethnic groups, among 
both males and females, and across all levels of incoming 
achievement. And yet, when these efforts were initiated, 
it was not known whether improvements in on-track 
rates would, in fact, pay dividends in terms of graduation 
rates and improved performance in subsequent years. 


DEFINITION OF ON-TRACK 

A student is considered “on-track” to graduate if he or she earns at least 
five full-year course credits (ten semester credits) and no more than one 
semester F in a core course (English, math, science, or social science) in 
their first year of high school. 
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FIGURE 1 


Ninth grade on-track rates improved most when schools acted on real time data about student absences and 
course performance 



Note: This figure represents first-time freshmen who are not receiving special education services and are not enrolled in a charter or alternative school. 


On-Track Rates Are Up, But Did 
Efforts to Improve Ninth Grade 
On-Track Rates Actually Lead to 
Higher Graduation Rates? 

Chicago’s efforts to improve student performance in 
ninth grade convincingly demonstrate that high schools 
can affect whether students fail ninth grade courses 
and are on- or off-track. What educators could not know 
when they started their work was whether getting more 
ninth graders on-track would subsequently improve 
those students’ likelihood of graduating high school 
on time or their performance in subsequent grades. If, 
for example, the on-track indicator simply identified 
those students who enter ninth grade disengaged or 
with very low skills —the students who are most likely 
to struggle throughout high school— then we would not 
expect rising ninth grade on-track rates to carry over 
into improved performance in later grades since those 
problems would still plague students in later years. 

There was good reason to believe that reducing 
course failure in ninth grade would improve students’ 
chances of graduating from high schools, given the 
strength of the relationship between ninth grade 
performance and eventual graduation. But there were 
also reasonable concerns that these efforts to reduce 
course failures in ninth grade might be misguided— 


that raising on-track rates may only be kicking the 
can down the road by delaying the onset of academic 
problems until later in students’ high school careers, 
and ultimately creating incentives for high schools to 
simply promote unqualified students. There have also 
been questions raised about whether the inclusion 
of on-track in CPS’s accountability system has led 
teachers to game the system, turning F’s into D’s or 
focusing on preventing students with only one F from 
failing additional courses. 

In addition, policymakers and educators often 
express the concern that there may be unintended 
consequences of these efforts. One concern is that 
schools may pay a price for increasing on-track rates 
in the form of lower test scores caused by keeping 
lower-performing or academically vulnerable students 
in school longer. Second, at a time when districts across 
the country are emphasizing the importance of college 
readiness, there has been concern that efforts aimed 
explicitly at mitigating course failure in ninth grade 
may send the wrong message to students and teachers 
alike that just passing classes is enough, thus under- 
mining a focus on college readiness. Finally, there 
have been doubts about whether these fairly low-cost 
efforts could do much to change outcomes in schools 
where so few students graduated that the schools were 
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called “dropout factories" OT among African American 
and Latino males whose graduation rates were under 
50 percent. 

Much of the debate rests on legitimate concerns that 
can be formulated as testable propositions. By compar- 
ing consecutive cohorts as they move from ninth grade 
into subsequent grades, we can test these concerns 
empirically. For example, if an increase in a high school’s 
on-track rate means that more of its students have been 
set on a path towards graduation, then we would expect 
to observe increases in that ninth grade cohort’s perfor- 
mance in tenth, eleventh, and twelfth grade, culminat- 
ing in a higher graduation rate among those students. If, 
however, off-track status is simply a marker of students 
who enter ninth grade disengaged or with very low skills, 
then we would expect students to fail in later grades, 
once they are no longer getting the additional attention 
aimed at ninth-graders. We can use the same approach 
to ask whether there is evidence that efforts to raise on- 
track rates in ninth grade led teachers to turn Fs into Ds, 
simply passing along failing students to the next grade. 

Improvements in Ninth Grade 
On-Track Rates Led to Higher 
Graduation Rates and Better 
Academic Outcomes for Students 

The dramatic improvements in on-track rates in Chicago 
provide an opportunity to answer these questions. A 
subset of 20 high schools showed substantial improve- 
ments in on -track rates as early as 2008 and 2009, so that 
sufficient time has elapsed to allow us to explore whether 
increases in ninth grade on-track rates have produced 
increases in high school graduation rates downstream. 
The report separately examines two groups of schools: 


1) primary movers, three schools (Juarez, Kenwood, and 
Steinmetz) that showed large improvements in on-track 
rates in 2008; and secondary movers, 17 additional 
schools that showed improvements in 2009. Other high 
schools showed improvements in subsequent years, 
but the 20 schools are the focus of the study since they 
improved ninth grade on-track rates early enough to ex- 
amine changes in four-year graduation rates as of 2013. 

Key Findings 

Improvements in ninth grade on-track at eariy-mover 
schoois were sustained in tenth and eieventh grade 
and foiiowed by a iarge increase in graduation rates. 

All three primary mover schools increased their on -track 
rates by at least 10 percentage points in 2008, compared 
with the prior three years. The following year, the rela- 
tionship between ninth grade on-track and tenth grade 
performance remained the same for those students as in 
prior years, even though many more freshman were now 
ending the year on-track. In 2011, after those students 
were in school for four years, graduation rates jumped at 
all three schools, with increases that ranged from eight 
to 20 percentage points. Among secondary movers, on- 
track rates increased, on average, by 11 percentage points 
in 2009, compared with the baseline cohorts; in 2012, 
on average, graduation rates at these schools jumped by 
13 percentage points (see Figure 2). These graduation 
rate improvements among the early movers cannot be 
attributed to changes in the incoming characteristics of 
students; incoming achievement levels (e.g., eighth grade 
test scores) were not significantly better than in prior 
cohorts. Rather, there is a strong relationship between 
the initial increase in ninth grade on-track rates and 
improved outcomes four years later. 


On-track rates improved more among African American males than among 
any other racial/ethnic gender subgroup, rising from 43 percent in 2005 
to 71 percent in 2013. 
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FIGURE 2 

In secondary mover schools, increases in 9th grade on-track rates persisted as that cohort progressed 
through high school 
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Note: This figure represents first-time freshmen at secondary mover schools without a special education status. Years refer to the spring of the school year. 
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Between 2005 and 2013, improvements in ninth grade 
on-track rates across the district were dramatic, 
sustained, and observed across a wide range of high 
schoois and among criticai subgroups— by race, by 
gender and across achievement ieveis. The benefits of 
getting on-track were greatest for students with the low- 
est incoming skills (see Tabie 1). Students with incom- 
ing EXPLORE scores less than 12— the bottom quartile 
of CPS students— had a 24.5 percentage point increase 
in their on-track rates. On-track rates improved more 
among African American males than among any other 
racial/ethnic gender subgroup, rising from 43 percent 
in 2005 to 71 percent in 2013. 

Furthermore, in all but one early mover schools, the 
improvements in ninth grade on-track rates continued 
to improve among subsequent cohorts (see Figure 3). 

In later years, more schools showed improvementsin 
ninth grade on-track rates and city wide on-track rates 
have continued to improve. In 2005, on-track rates 
varied widely across high schools; by 2013 only a hand- 
ful of schools had on-track rates under 70 percent 
(see Figure 4). System shifts - large, consistent im- 
provements that create lasting change across subgroups 
and considerably reduce variation in outcomes across 
schools - are rare in education. This is a one of those 
rare cases. 


Improvements in on-track were accompanied by 
across-the-board improvements in grades. On-track 
improvement was not just a result of teachers giving stu- 
dents a grade of “D” instead of an “F.” Grades improved 
at all ends of the achievement spectrum. In the 17 sec- 
ondary-mover schools, there was a 10 percentage point 
increase in the proportion of freshmen with B’s or better 
and a 12 percentage point increase in the percentage of 
students with no F’s. The portion of students with D’s or 
F’s declined significantly, from 33 to 22 percent, while 
the percentage of students with B’s or better increased 
from 28 to 37 percent. Thus, there is little evidence, 
on average, that the increase in on-track rates in these 
schools was driven by simply focusing on turning F’s into 
D’s or on trying to move students at the margins. 

Increasing ninth grade on-track rates did not nega- 
tively affect high schools’ average ACT scores— 
despite the fact that many more students with weaker 
incoming skills made it to junior year to take the test. 

ACT scores remained very close to what they were 
before on-track rates improved, which means that 
the average growth from EXPLORE to ACT remained 
the same or increased, even though more students— 
including many students with weaker incoming skills— 
were taking the ACT. 
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TABLE 1 

On-track rates improved for all groups of CPS students, but African American males improved the most 


On-Track Rates by 
Race/Ethnicity and Gender 

Percent of 
Entering Class 

2005 

On-Track Rate 

2013 

On-Track Rate 

Change 

African American Males 

19% 

43.1% 

71.4% 

+28.3 

African American Females 

21% 

61.2% 

82.3% 

+21.1 

Latino Males 

25% 

52.1% 

77.4% 

+25.3 

Latina Females 

24% 

66.4% 

85.6% 

+19.2 

White Males 

5% 

66.8% 

87.2% 

+20.4 

White Females 

5% 

79.6% 

93.1% 

+13.5 


On-Track Rates by 





EXPLORE Categories 





<12 

12% 

41.6% 

66.1% 

+24.5 

12-13 

20% 

52.8% 

71.2% 

+18.4 

14-16 

32% 

68.8% 

86.3% 

+17.5 

17+ 

30% 

84.6% 

94.9% 

+10.3 


Note: This figure represents first time freshmen not in charter or alternative schools without a special education status. Years refer to the spring of the school 
year. Entering class compositions are of the 2013 class. Approximately 7% of freshmen are missing EXPLORE scores and are not included in EXPLORE sub-groups. 


FIGURE 3 

Almost all primary and secondary mover schools continued to improve through 2013 



10% 


n 1 r 

40% 50% 60% 

Freshman On-Track Rate 


70% 


I 2004-06 Baseline 


I 2009 Increase 


I 2013 Increase 


Note: This figure represents first-time freshmen who are not receiving special education services and are not enrolled in a charter or alternative school. Years refer to the 
spring of the school year. Three secondary mover schools, Chicago Discovery, School of Entrepreneurship, and School of the Arts closed after 2009 and are not shown. 
‘Uplift's freshman on-track rate increased by 18% in 2009, then decreased by 8% in 2013, giving the school a total increase of 83% since 2004-06. 
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FIGURE 4 

While only 22 high schools had on-track rates above 70% in 2005, by 2013, nearly 90% of high schools had on-track 
rates that high. 
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Note: This figure represents first-time freshmen who are not receiving special education services and are enrolled in schools that were open in both spring 
2005 and spring 2013, respectively. 
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Interpretive Summary 

There is good reason for skepticism about educational reforms. 
Practitioners have seen reforms come and go and are naturally dubious 
about the potential of “the next new thing” to take root and make real 
change. Researchers are also very skeptical of findings with big effects. 
And yet, CPS’s focus on on-track moved an entire system. The evidence 
in this brief shows that the improvements in ninth grade on-track rates 
can result in dramatic improvements in graduation rates three years 
later. It is hard to overstate the significance of these findings. 


The on-track initiative reframed the problem of schooi 
dropout from an outcome that is outside the controi 
of educators to one that can be managed through 
effective schooi-based strategies. What is striking 
is how substantial improvements in performance can 
result from a targeted approach that also allows schools 
latitude in their strategy. In contrast to the common as- 
sumption that school dropout is an intractable problem 
in urban schools, the improvements in on-track and 
graduation rates in the primary- and secondary-mover 
schools suggest that students’ performance is actually 
highly malleable. 

Ninth grade is a pivotai year that provides a unique 
intervention point to prevent school dropout. What is 
clear is that no matter how a school increases on-track 


rates in ninth grade, graduation rates improve three 
years later. When schools concentrate their efforts on 
helping students make a successful transition to ninth 
grade, it results in dramatic increases in graduation. 

Chicago’s focus on on-track provides an important 
case study of the use of data to buiid the capacity of 
high school educators to manage complex problems 
and create systems of continuous improvement. The 
success of this initiative raises unanswered questions 
about how data use changed educators’ behaviors, why 
some schools continued to improve and others did not, 
and, in particular, why this specific approach was so 
effective in generating improvements in the system’s 
lower performing schools. All of these questions would 
be important subjects for further research. 


In contrast to the common assumption that school dropout is an intractable 
problem in urban schools, the improvements in on-track and graduation 
rates in the primary- and secondary-mover schools suggest that students’ 
performance is actually highly malleable. 


ONTgAC^ 
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■ 

I The University of Chicago Consortium on Chicago 

r School Research (UChicago CCSR) conducts research of high technical 
quality that can inform and assess policy and practice in the Chicago 
Public Schools. We seek to expand communication among researchers, 
policymakers, and practitioners as we support the search for solutions 
to the problems of school reform. UChicago CCSR encourages the use 
of research in policy action and improvement of practice, but does 
not argue for particular policies or programs. Rather, we help to build 
capacity for school reform by identifying what matters for student 
success and school improvement, creating critical indicators to chart 
progress, and conducting theory-driven evaluation to identify how 
programs and policies are working. 
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